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ABSTRACT 

Embedded  training  systems  have  several  unique  requirements  as  compared  to  more  traditional  training 
systems.  For  the  most  part,  these  requirements  stem  from  the  fact  that  traditional  instructors  and  other 
technical  support  personnel  are  not  typically  available  in  the  operational  environment.  Hence,  embedded 
training  must  be  essentially  stand-alone  systems.  This  requirement  means  that  embedded  training  systems 
must  be  highly  automated  and  intelligent. 

As  a  starting  point,  it  must  be  noted  that  developing  effective  embedded  training  systems  should  hinge  on 
sound  research  into  how  people  learn.  In  addition,  modern  technology  offers  a  variety  of  capabilities  that 
were  not  available  even  5  years  ago.  Taken  together,  principles  of  learning  supported  by  technology  can 
yield  effective  embedded  training  systems.  The  sections  that  follow  lay  out  the  components  of  an  effective 
embedded  training  system.  It  begins  with  tenets  from  scenario-based  training  and  then  adds  components 
of  intelligent  tutoring. 


1.0  SCIENCE  BEHIND  SCENARIO-BASED  TRAINING 

The  challenge  of  building  adaptive  expertise  is  a  major  thrust  in  most  modem  Military  training.  Over  the 
past  thirty  years,  the  development  of  expertise  has  been  the  focus  of  much  study.  Based  on  this  body  of 
work,  theoretical  justification  has  been  amassed  for  the  proposition  that  providing  trainees  with  realistic 
scenarios  can  be  an  effective  means  to  accelerate  the  development  of  expertise.  Such  justification  can  be 
found  in  several  lines  of  research  into  how  people  learn.  In  this  regard,  Glaser  (1989)  argued  that 
“beginners’  knowledge  is  spotty,  consisting  of  isolated  definitions  and  superficial  understandings  of 
central  terms  and  concepts”  (p.  272).  Over  time,  these  items  of  information  become  more  structured  and 
are  integrated  with  past  organizations  of  knowledge,  eventually  being  stored  as  condition-action  rules  in 
memory.  Thus,  through  acquisition  of  domain  knowledge,  experts  build  up  a  repertoire  of  instances — or 
patterns— indexing  them  in  such  a  way  that  they  are  rapidly  accessible  when  triggered  by  environmental 
cues  (Chi,  Glaser,  &  Farr,  1988,  p.  xvii;  Gobet  &  Simon,  1996;  Logan,  1988).  Experts  also  appear  to 
“chunk”  information  into  related  packets,  and  recognize  meaningful  patterns  in  the  problem  space,  a 
finding  that  has  been  replicated  across  domains  (Egan  &  Schwartz,  1979;  DeGroot,  1965). 

Based  on  this  and  other  work,  principles  for  the  design  of  effective  learning  systems  are  numerous.  Eirst, 
the  notion  that  learning  should  occur  in  a  meaningful  or  relevant  context  is  supported  by  these  theoretical 
positions.  Second,  expert  knowledge  is  conditionalized  (i.e.,  specific  to  a  context)  and  develops  over  time 
through  repeated  exposure  to  instances  of  the  task  (which  is  another  way  of  saying  experience).  New 
learning,  in  turn,  must  be  integrated  into  this  existing  world  knowledge. 

Research  over  the  past  20  years  has  produced  results  that  can  enable  the  development  of  learning 
environments  that  are  consistent  with  principles  noted  above.  Eor  example,  simulations  that  afford  trainees 
opportunities  for  practice  and  experimentation  have  proven  very  effective  in  a  variety  of  contexts  (Stout  et 
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al.,  2008).  However,  the  effectiveness  of  any  training  system — simulation-based  or  otherwise — depends 
on  how  well  the  system  is  designed  and  whether  it  embodies  sound  instructional  features  (Salas  & 
Cannon-Bowers,  2000).  In  the  case  of  simulation,  much  of  the  instructional  power  is  actually  attributable 
to  the  nature  of  the  scenario  that  is  expressed  in  the  simulated  environment.  Hence  it  is  imperative  to 
understand  how  to  design  and  implement  effective  scenarios  as  a  means  to  optimize  scenario-based 
training. 


2.0  A  FRAMEWORK  FOR  DESIGNING  EFFECTIVE  SCENARIO-BASED 
TRAINING  (SBT) 

Cannon-Bowers  et  al.  (1998)  proposed  an  overarching  framework  to  describe  the  SBT  process  based  on 
extensive  experience  in  training  Navy  combat  teams.  This  process  has  been  modified  over  the  years;  the 
updated  version  can  be  seen  in  Figure  1 . 


[Insert  Figure  1  about  here] 

The  SBT  process  shown  in  Figure  1  provides  an  organizing  framework  for  couching  a  discussion  of  SBT. 
The  following  sections  provide  more  detail  about  the  SBT  process  and  describe  how  each  step  in  the 
process  contributes  a  crucial  component  that  optimizes  the  overall  system  performance  and  effectiveness. 

2.1  Task  Analysis  and  Learning  Objectives 

It  has  long  been  acknowledged  that  a  detailed  task  analysis  is  a  first  essential  step  in  developing  any 
training  system.  It  stands  to  reason  that  the  targeted  task  and  domain  must  be  well  understood  before 
training  can  be  developed,  and  job/task  analysis  methods  to  accomplish  this  have  been  existence  for  many 
years  (Annett  &  Stanton,  2006).  However,  traditional  task/domain  analysis  processes  are  deficient  for 
several  reasons. 

First,  more  attention  is  being  paid  to  higher  order  skills,  especially  critical  thinking,  decision-making  and 
problem  solving.  This  has  resulted  in  efforts  to  better  understand  how  experts  perform  in  realistic 
environments,  including  specification  of  the  tacit  or  implicit  knowledge  that  is  involved — such  knowledge 
is  crucial  to  task  performance,  but  not  well  articulated  by  the  experts  themselves  (Cianciolo,  Matthew, 
Sternberg,  &  Wagner,  2006).  Hence,  cognitive  task  analysis — which  represent  less  direct  methods  of 
eliciting  knowledge  from  experts— have  been  developed  in  recent  years  as  a  means  to  understand  more 
completely  how  an  expert  would  perform  the  task. 

Next,  task  analysis  data  must  cast  into  a  set  of  observable,  measurable  learning  objectives  for  an  exercise 
or  training  event.  This  step  may  seem  obvious,  but  is  often  overlooked;  it  is  crucial  for  several  reasons. 
First,  it  helps  to  organize  the  scenario  and  ensure  that  the  most  important  knowledge  and  skills  are 
addressed.  Second,  it  helps  to  ensure  that  the  exercise  is  targeted  appropriately  for  the  learners — so  that  it 
is  neither  too  easy  (and  hence  not  challenging),  nor  too  overwhelming  so  that  trainees  cannot  gain 
maximum  benefit.  In  addition,  formally  stating  learning  objectives  at  the  onset  of  an  exercise  is  important 
so  that  it  is  clear  what  is  to  be  covered  and  that  trainees  are  practicing  a  manageable  number  of  skills. 
Finally,  the  learning  objectives  for  an  exercise  or  event  should  lead  directly  to  a  set  of  measurable 
outcomes  (more  will  be  said  about  performance  measurement  in  a  later  section). 


3.0  SCENARIO:  EVENTS  AND  STORY 

Once  the  learning  objectives  have  been  established,  they  can  then  be  used  as  input  to  develop  scenarios, 
which  form  the  basis  of  a  training  exercise.  With  respect  to  developing  effective  scenarios,  past 
researchers  have  conceptualized  this  process  in  terms  of  embedding  events  into  scenarios  that  represent  the 
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learning  objectives.  An  event,  in  this  context,  is  any  stimulus  condition  that  is  purposely  scripted  into  a 
scenario  in  order  to  elicit  a  particular  response.  Scenario  events  have  also  been  conceptualized  as 
triggers — specific  scenario  conditions  that  will  allow  the  trainee  to  practice  the  targeted  learning 
objectives  (Fowlkes  et  al.,  1998)  as  a  means  to  demonstrate  proficiency  and/or  as  a  basis  for  feedback. 
Hence,  the  scenario  events  are  the  basis  of  a  trainee’s  practice  opportunities. 

The  scenario  in  SET  also  serves  another  purpose;  that  is,  to  provide  a  context  or  narrative  that  ties  events 
together.  In  this  sense,  the  scenario  or  story  also  serves  a  motivational  purpose  because  it  engages  the 
trainee  in  a  realistic  context.  Researchers  have  recently  begun  to  theorize  that  narrative  elements  may 
actually  enhance  learning  by  helping  guide  trainees  through  the  system  (Ironside,  2006).  Further,  recent 
research  into  concepts  such  as  immersion  and  presence  seems  to  indicate  that  learning  can  be  enhanced 
when  trainees  are  psychologically  engaged  in  the  scenario  (Whitmer  &  Singer,  ).  Stories  and  story-based 
learning  also  help  to  ensure  that  the  experiences  trainees  gain  in  training  (as  opposed  to  the  real  world)  are 
authentic — that  is,  they  are  rich,  faithful  representations  of  the  world  that  will  enable  the  trainee  to  transfer 
his/her  virtual  experience  into  the  real  environment.  This  includes  the  affective  or  emotional  aspects  as 
well  as  more  cognitive  and  behavioral. 

3.1  Performance  Measurement/Metrics/Diagnosis 

Once  scenario  events  are  scripted  and  instructional  strategies  selected,  the  next  step  in  the  SET 
development  process  is  to  specify  the  performance  measures  that  will  be  implemented  to  assess  trainee 
actions  and  behaviors.  In  general,  performance  measurement  is  challenging  in  high  performance 
environments  because  it  is  difficult  to  assess  complex,  and  often  times  unobservable,  performance.  For 
example,  in  many  cases,  the  situation  is  unfolding  rapidly  and  there  are  many  things  going  on  at  once  (this 
is  particularly  true  in  team  situations).  Therefore,  simply  sorting  tracking  and  sorting  through  trainees’ 
performance  is  complicated. 

It  is  also  often  difficult  to  evaluate  performance  since  it  is  impossible  to  track  a  trainee’s  thought  process 
directly.  Hence,  instructors  may  not  be  able  to  comment  on  the  process  that  the  trainee  used  to  arrive  at  a 
conclusion  or  action.  Further,  establishing  standards  of  performance — this  is,  a  set  of  criteria  that  describe 
desired  performance — is  not  feasible  in  uncertain,  ambiguous  decision  making  situations  where  there  are 
many  possible  ways  to  solve  a  problem. 

Perhaps  the  most  important  function  of  performance  measurement  is  the  ability  to  diagnose  the  causes  of 
observed  performance.  That  is,  it  is  not  sufficient  to  say  that  a  trainee  performed  poorly— in  order  to 
intervene  so  that  performance  can  improve,  it  is  essential  to  understand  why  the  behavior  was 
demonstrated.  For  example,  if  the  student’s  performance  can  be  attributed  to  a  lack  fundamental 
knowledge,  this  would  suggest  one  remediation  strategy,  while  a  deficiency  in  skills  (e.g.,  communication) 
would  suggest  an  entirely  different  one. 

3.2  Instructional  Strategies 

SET  environments  provide  a  context  in  which  learning  can  occur,  but  they  are  not  training  systems  in  and 
of  themselves,  without  additional  elements  displayed  in  Figure  1.  The  establishment  of  instructional 
strategies  that  optimize  learning  is  of  primary  importance  here;  there  are  several  possible  approaches  to 
embedding  instruction  in  SET.  For  example,  instructional  decisions  can  be  made  regarding  the  difficulty 
of  tasks  presented  to  trainees,  the  form  and  timing  of  feedback  (more  will  be  said  about  this  in  a  later 
section),  the  nature  of  hints  and  cues  provided  to  trainees,  the  spacing  of  practice  opportunities,  and  the 
like.  Consideration  of  the  nature  of  the  learning  objectives,  as  well  as  the  trainees’  level  of  mastery,  are 
critically  important  in  this  regard. 
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3.3  Feedback  &  Remediation 

As  noted,  feedback  is  an  essential  element  in  SBT,  as  it  is  in  all  forms  of  training.  Feedback  provides 
trainees  with  a  detailed  understanding  of  their  performance  and  what  needs  to  be  corrected  in  order  to 
enhance  subsequent  performance.  Much  has  been  written  about  feedback  in  training,  including  issues 
such  as:  when  and  how  often  to  give  feedback,  the  format  of  feedback  (e.g.,  directive  or  reflective),  the 
specificity  of  feedback,  and  who  provides  feedback  (instructors  or  trainees  themselves.  The  literature 
provides  much  useful  guidance  on  how  best  to  implement  feedback  mechanisms  in  training.  In  SBT, 
feedback  is  a  primary  mechanism  for  imparting  targeted  objectives;  hence  it  must  be  carefully  developed 
and  implemented. 

3.4  Learning  Management 

A  final  step  in  the  SBT  process  is  to  record  trainee  performance,  and  use  that  information  to  guide  future 
training  exercises.  In  operational  environments  this  is  often  accomplished  informally,  rendering 
subsequent  training  sessions  less  effective  than  they  could  be.  The  requirement  is  to  accurately  identify 
and  record  the  trainees’  level  of  mastery  so  that  future  training  events  can  be  tailored  to  their  needs. 
Otherwise,  the  possibility  of  repeating  practice  on  learning  objectives  that  have  already  been  mastered  is 
high.  It  is  necessary  in  this  regard  that  performance  outcomes  from  SBT  exercises  be  carefully  recorded 
so  that  they  can  inform  subsequent  training. 

3.5  Intelligent  Tutoring 

Traditional  intelligent  tutoring  systems  seek  to  automatically  track,  interpret  and  act  upon  real  time  trainee 
performance  data.  The  history  of  intelligent  tutoring  spans  the  last  20  years,  and  for  the  most  part, 
successful  tutors  have  been  implemented  for  static  tasks  (such  as  math,  programming).  More  complex 
environments  have  typically  exceeded  the  ability  of  systems  to  track  and  interpret  data.  Further,  models  of 
complex  performance  have  typically  not  been  available.  Modern  technology  is  providing  new  capabilities 
that  may  be  able  to  address  these  deficiencies. 

3.5.1  Components  of  Intelligent  Tutoring 

Traditional  intelligent  tutoring  systems  have  several  key  components.  These  include: 

•  Expert  model — detailed  model  of  expert  performance  used  as  a  means  to  compare  observed 
student  performance. 

•  Student  model — executable  model  of  student  performance  that  is  built  up  over  time  based  on 
observed  (on-going)  student  performance. 

•  Instructional  model — model  that  guides  the  specification  of  what  should  be  done 
(feedback/remediation)  in  response  to  various  student  performances 

•  Tracking  mechanisms — instrumentation  that  allows  collection  of  a  variety  of  performance  data 
including,  keystrokes,  eye  gaze,  head  movements,  body  movements,  verbal  responses,  and/or 
instructor  input. 

•  Simulation — typically,  intelligent  tutoring  for  dynamic  tasks  are  based  on  some  type  of 
simulation. 

These  components  are  compatible  with  the  scenario-based  training  model  offered  above.  When  combined, 
these  capabilities  have  the  potential  to  provide  Intelligent  Embedded  Training  systems  that  can  be 
implemented  in  operational  environments — without  much  support  from  instructors  or  other  technicians. 
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A  vision  of  intelligent  embedded  training  includes  automated  scenario  generation  capabilities  that  allow 
instructors  to  easily  develop  new  scenarios  that  correspond  to  changing  mission  requirements. 
Importantly,  this  process  would  also  result  in  specification  of  performance  measures  and  diagnostic 
routines,  most  likely  developed  through  an  interactive  authoring  dialog  with  the  instructor.  Once  scripted, 
a  new  scenario  would  be  easily  implemented  in  the  embedded  training  system  so  that  trainees  could 
automatically  receive  feedback  about  their  performance. 


4.0  THE  WAY  AHEAD 

Early  attempts  to  develop  Intelligent  Embedded  Training  met  with  only  limited  success,  and  few,  if  any 
such  systems  are  in  use.  However,  emerging  technologies  offer  promise  in  this  regard,  and  computing 
power  has  grown  to  the  point  where  even  data  intensive  systems  are  feasible. 
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